Robotic system with indication of boundary for robotic arm

ABSTRACT

Certain aspects relate to systems and techniques for surgical robotic arm setup. In one aspect, there is provided a system including a first robotic arm configured to manipulate a medical instrument, a processor, and a memory. The processor may be configured to: determine a minimum stroke length of the first robotic arm that allows advancing of the medical instrument by the first robotic arm to reach a target region from an access point via a path, determine a boundary for an initial pose of the first robotic arm based on the minimum stroke length and a mapping stored in the memory, and during an arm setup phase prior to performing a procedure, provide an indication of the boundary during movement of the first robotic arm.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/568,733, filed Oct. 5, 2017, which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The systems and methods disclosed herein are directed to robotic armsetup, and more particularly to providing an indication of a boundaryfor an initial pose of a robotic arm of a robotic system.

BACKGROUND

Medical procedures such as endoscopy (e.g., bronchoscopy) may involvethe insertion of a medical tool into a patient's luminal network (e.g.,airways) for diagnostic and/or therapeutic purposes. Surgical roboticsystems may be used to control the insertion and/or manipulation of themedical tool during a medical procedure. The surgical robotic system maycomprise at least one robotic arm including a manipulator assembly whichmay be used to control the positioning of the medical tool prior to andduring the medical procedure.

SUMMARY

The systems, methods and devices of this disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

In one aspect, there is provided a system comprising: a first roboticarm configured to manipulate a medical instrument; a processor; and amemory storing a mapping of an anatomy of a patient. The mapping maycomprise data regarding (i) a target region within the anatomy and (ii)a path from an access point of the patient to the target region. Thememory may further store computer-executable instructions, that whenexecuted, cause the processor to: determine a minimum stroke length ofthe first robotic arm that allows advancing of the medical instrument bythe first robotic arm to reach the target region from the access pointvia the path, determine a boundary for an initial pose of the firstrobotic arm based on the minimum stroke length and the mapping, andduring an arm setup phase prior to performing a procedure, provide anindication of the boundary during movement of the first robotic arm.

In another aspect, there is provided a non-transitory computer readablestorage medium having stored thereon instructions that, when executed,cause at least one computing device to: determine a minimum strokelength of a first robotic arm that allows advancing of a medicalinstrument by the first robotic arm to a target region based on amapping of an anatomy of a patient, the mapping comprising dataregarding (i) the target region within the anatomy and (ii) a path froman access point of the patient to the target region, the medicalinstrument advanced to reach the target region from the access point viathe path; determine a boundary for an initial pose of the first roboticarm based on the minimum stroke length and the mapping; and during anarm setup phase prior to performing a procedure, provide an indicationof the boundary during movement of the first robotic arm.

In yet another aspect, there is provided a method of positioning a firstrobotic arm, comprising: determining a minimum stroke length of thefirst robotic arm that allows advancing of a medical instrument by thefirst robotic arm to reach a target region based on a mapping of ananatomy of a patient, the mapping comprising data regarding (i) thetarget region within the anatomy and (ii) a path from an access point ofthe patient to the target region, the medical instrument advanced toreach the target region from the access point via the path; determine aboundary for an initial pose of the first robotic arm based on theminimum stroke length and the mapping; and during an arm setup phaseprior to performing a procedure, provide an indication of the boundaryduring movement of the first robotic arm.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements.

FIG. 1 illustrates an embodiment of a cart-based robotic system arrangedfor diagnostic and/or therapeutic bronchoscopy procedure(s).

FIG. 2 depicts further aspects of the robotic system of FIG. 1.

FIG. 3 illustrates an embodiment of the robotic system of FIG. 1arranged for ureteroscopy.

FIG. 4 illustrates an embodiment of the robotic system of FIG. 1arranged for a vascular procedure.

FIG. 5 illustrates an embodiment of a table-based robotic systemarranged for a bronchoscopy procedure.

FIG. 6 provides an alternative view of the robotic system of FIG. 5.

FIG. 7 illustrates an example system configured to stow robotic arm(s).

FIG. 8 illustrates an embodiment of a table-based robotic systemconfigured for a ureteroscopy procedure.

FIG. 9 illustrates an embodiment of a table-based robotic systemconfigured for a laparoscopic procedure.

FIG. 10 illustrates an embodiment of the table-based robotic system ofFIGS. 5-9 with pitch or tilt adjustment.

FIG. 11 provides a detailed illustration of the interface between thetable and the column of the table-based robotic system of FIGS. 5-10.

FIG. 12 illustrates an exemplary instrument driver.

FIG. 13 illustrates an exemplary medical instrument with a pairedinstrument driver.

FIG. 14 illustrates an alternative design for an instrument driver andinstrument where the axes of the drive units are parallel to the axis ofthe elongated shaft of the instrument.

FIG. 15 depicts a block diagram illustrating a localization system thatestimates a location of one or more elements of the robotic systems ofFIGS. 1-10, such as the location of the instrument of FIGS. 13 and 14,in accordance to an example embodiment.

FIG. 16 illustrates an embodiment of a surgical robotic system arrangedfor diagnostic and/or therapeutic bronchoscopy procedure(s) inaccordance with aspects of this disclosure.

FIG. 17A is a flow-chart which illustrates features of an example setupprocedure for a medical procedure in accordance with aspects of thisdisclosure.

FIG. 17B is a flow-chart which illustrates features of another examplesetup procedure for a medical procedure in accordance with aspects ofthis disclosure.

FIG. 18 illustrates an embodiment of a bronchoscope which may be used inaccordance with aspects of this disclosure.

FIG. 19 is a flow-chart which illustrates another example of a setupprocedure for a bronchoscopy procedure in accordance with aspects ofthis disclosure.

FIG. 20 is a flow-chart which illustrates an example methodology forsimulating a medical procedure in accordance with aspects of thisdisclosure.

DETAILED DESCRIPTION 1. Overview

Aspects of the present disclosure may be integrated into arobotically-enabled medical system capable of performing a variety ofmedical procedures, including both minimally invasive, such aslaparoscopy, and non-invasive, such as endoscopy, procedures. Amongendoscopy procedures, the system may be capable of performingbronchoscopy, ureteroscopy, gastroscopy, etc.

In addition to performing the breadth of procedures, the system mayprovide additional benefits, such as enhanced imaging and guidance toassist the physician. Additionally, the system may provide the physicianwith the ability to perform the procedure from an ergonomic positionwithout the need for awkward arm motions and positions. Still further,the system may provide the physician with the ability to perform theprocedure with improved ease of use such that one or more of theinstruments of the system can be controlled by a single user.

Various embodiments will be described below in conjunction with thedrawings for purposes of illustration. It should be appreciated thatmany other implementations of the disclosed concepts are possible, andvarious advantages can be achieved with the disclosed implementations.Headings are included herein for reference and to aid in locatingvarious sections. These headings are not intended to limit the scope ofthe concepts described with respect thereto. Such concepts may haveapplicability throughout the entire specification.

A. Robotic System—Cart

The robotically-enabled medical system may be configured in a variety ofways depending on the particular procedure. FIG. 1 illustrates anembodiment of a cart-based robotically-enabled system 10 arranged for adiagnostic and/or therapeutic bronchoscopy procedure. During abronchoscopy, the system 10 may comprise a cart 11 having one or morerobotic arms 12 to deliver a medical instrument, such as a steerableendoscope 13, which may be a procedure-specific bronchoscope forbronchoscopy, to a natural orifice access point (i.e., the mouth of thepatient positioned on a table in the present example) to deliverdiagnostic and/or therapeutic tools. As shown, the cart 11 may bepositioned proximate to the patient's upper torso in order to provideaccess to the access point. Similarly, the robotic arms 12 may beactuated to position the bronchoscope relative to the access point. Thearrangement in FIG. 1 may also be utilized when performing agastro-intestinal (GI) procedure with a gastroscope, a specializedendoscope for GI procedures. FIG. 2 depicts an example embodiment of thecart in greater detail.

With continued reference to FIG. 1, once the cart 11 is properlypositioned, the robotic arms 12 may insert the steerable endoscope 13into the patient robotically, manually, or a combination thereof. Asshown, the steerable endoscope 13 may comprise at least two telescopingparts, such as an inner leader portion and an outer sheath portion, eachportion coupled to a separate instrument driver from the set ofinstrument drivers 28, each instrument driver coupled to the distal endof an individual robotic arm. This linear arrangement of the instrumentdrivers 28, which facilitates coaxially aligning the leader portion withthe sheath portion, creates a “virtual rail” 29 that may be repositionedin space by manipulating the one or more robotic arms 12 into differentangles and/or positions. The virtual rails described herein are depictedin the Figures using dashed lines, and accordingly the dashed lines donot depict any physical structure of the system. Translation of theinstrument drivers 28 along the virtual rail 29 telescopes the innerleader portion relative to the outer sheath portion or advances orretracts the endoscope 13 from the patient. The angle of the virtualrail 29 may be adjusted, translated, and pivoted based on clinicalapplication or physician preference. For example, in bronchoscopy, theangle and position of the virtual rail 29 as shown represents acompromise between providing physician access to the endoscope 13 whileminimizing friction that results from bending the endoscope 13 into thepatient's mouth.

The endoscope 13 may be directed down the patient's trachea and lungsafter insertion using precise commands from the robotic system untilreaching the target destination or operative site. In order to enhancenavigation through the patient's lung network and/or reach the desiredtarget, the endoscope 13 may be manipulated to telescopically extend theinner leader portion from the outer sheath portion to obtain enhancedarticulation and greater bend radius. The use of separate instrumentdrivers 28 also allows the leader portion and sheath portion to bedriven independent of each other.

For example, the endoscope 13 may be directed to deliver a biopsy needleto a target, such as, for example, a lesion or nodule within the lungsof a patient. The needle may be deployed down a working channel thatruns the length of the endoscope to obtain a tissue sample to beanalyzed by a pathologist. Depending on the pathology results,additional tools may be deployed down the working channel of theendoscope for additional biopsies. After identifying a nodule to bemalignant, the endoscope 13 may endoscopically deliver tools to resectthe potentially cancerous tissue. In some instances, diagnostic andtherapeutic treatments may need to be delivered in separate procedures.In those circumstances, the endoscope 13 may also be used to deliver afiducial to “mark” the location of the target nodule as well. In otherinstances, diagnostic and therapeutic treatments may be delivered duringthe same procedure.

The system 10 may also include a movable tower 30, which may beconnected via support cables to the cart 11 to provide support forcontrols, electronics, fluidics, optics, sensors, and/or power to thecart 11. Placing such functionality in the tower 30 allows for a smallerform factor cart 11 that may be more easily adjusted and/orre-positioned by an operating physician and his/her staff. Additionally,the division of functionality between the cart/table and the supporttower 30 reduces operating room clutter and facilitates improvingclinical workflow. While the cart 11 may be positioned close to thepatient, the tower 30 may be stowed in a remote location to stay out ofthe way during a procedure.

In support of the robotic systems described above, the tower 30 mayinclude component(s) of a computer-based control system that storescomputer program instructions, for example, within a non-transitorycomputer-readable storage medium such as a persistent magnetic storagedrive, solid state drive, etc. The execution of those instructions,whether the execution occurs in the tower 30 or the cart 11, may controlthe entire system or sub-system(s) thereof. For example, when executedby a processor of the computer system, the instructions may cause thecomponents of the robotics system to actuate the relevant carriages andarm mounts, actuate the robotics arms, and control the medicalinstruments. For example, in response to receiving the control signal,the motors in the joints of the robotics arms may position the arms intoa certain posture.

The tower 30 may also include a pump, flow meter, valve control, and/orfluid access in order to provide controlled irrigation and aspirationcapabilities to system that may be deployed through the endoscope 13.These components may also be controlled using the computer system oftower 30. In some embodiments, irrigation and aspiration capabilitiesmay be delivered directly to the endoscope 13 through separate cable(s).

The tower 30 may include a voltage and surge protector designed toprovide filtered and protected electrical power to the cart 11, therebyavoiding placement of a power transformer and other auxiliary powercomponents in the cart 11, resulting in a smaller, more moveable cart11.

The tower 30 may also include support equipment for the sensors deployedthroughout the robotic system 10. For example, the tower 30 may includeopto-electronics equipment for detecting, receiving, and processing datareceived from the optical sensors or cameras throughout the roboticsystem 10. In combination with the control system, such opto-electronicsequipment may be used to generate real-time images for display in anynumber of consoles deployed throughout the system, including in thetower 30. Similarly, the tower 30 may also include an electronicsubsystem for receiving and processing signals received from deployedelectromagnetic (EM) sensors. The tower 30 may also be used to house andposition an EM field generator for detection by EM sensors in or on themedical instrument.

The tower 30 may also include a console 31 in addition to other consolesavailable in the rest of the system, e.g., console mounted on top of thecart. The console 31 may include a user interface and a display screen,such as a touchscreen, for the physician operator. Consoles in system 10are generally designed to provide both robotic controls as well aspre-operative and real-time information of the procedure, such asnavigational and localization information of the endoscope 13. When theconsole 31 is not the only console available to the physician, it may beused by a second operator, such as a nurse, to monitor the health orvitals of the patient and the operation of system, as well as provideprocedure-specific data, such as navigational and localizationinformation. In other embodiments, the console 30 is housed in a bodythat is separate from the tower 30.

The tower 30 may be coupled to the cart 11 and endoscope 13 through oneor more cables or connections (not shown). In some embodiments, thesupport functionality from the tower 30 may be provided through a singlecable to the cart 11, simplifying and de-cluttering the operating room.In other embodiments, specific functionality may be coupled in separatecabling and connections. For example, while power may be providedthrough a single power cable to the cart, the support for controls,optics, fluidics, and/or navigation may be provided through a separatecable.

FIG. 2 provides a detailed illustration of an embodiment of the cartfrom the cart-based robotically-enabled system shown in FIG. 1. The cart11 generally includes an elongated support structure 14 (often referredto as a “column”), a cart base 15, and a console 16 at the top of thecolumn 14. The column 14 may include one or more carriages, such as acarriage 17 (alternatively “arm support”) for supporting the deploymentof one or more robotic arms 12 (three shown in FIG. 2). The carriage 17may include individually configurable arm mounts that rotate along aperpendicular axis to adjust the base of the robotic arms 12 for betterpositioning relative to the patient. The carriage 17 also includes acarriage interface 19 that allows the carriage 17 to verticallytranslate along the column 14.

The carriage interface 19 is connected to the column 14 through slots,such as slot 20, that are positioned on opposite sides of the column 14to guide the vertical translation of the carriage 17. The slot 20contains a vertical translation interface to position and hold thecarriage at various vertical heights relative to the cart base 15.Vertical translation of the carriage 17 allows the cart 11 to adjust thereach of the robotic arms 12 to meet a variety of table heights, patientsizes, and physician preferences. Similarly, the individuallyconfigurable arm mounts on the carriage 17 allow the robotic arm base 21of robotic arms 12 to be angled in a variety of configurations.

In some embodiments, the slot 20 may be supplemented with slot coversthat are flush and parallel to the slot surface to prevent dirt andfluid ingress into the internal chambers of the column 14 and thevertical translation interface as the carriage 17 vertically translates.The slot covers may be deployed through pairs of spring spoolspositioned near the vertical top and bottom of the slot 20. The coversare coiled within the spools until deployed to extend and retract fromtheir coiled state as the carriage 17 vertically translates up and down.The spring-loading of the spools provides force to retract the coverinto a spool when carriage 17 translates towards the spool, while alsomaintaining a tight seal when the carriage 17 translates away from thespool. The covers may be connected to the carriage 17 using, forexample, brackets in the carriage interface 19 to ensure properextension and retraction of the cover as the carriage 17 translates.

The column 14 may internally comprise mechanisms, such as gears andmotors, that are designed to use a vertically aligned lead screw totranslate the carriage 17 in a mechanized fashion in response to controlsignals generated in response to user inputs, e.g., inputs from theconsole 16.

The robotic arms 12 may generally comprise robotic arm bases 21 and endeffectors 22, separated by a series of linkages 23 that are connected bya series of joints 24, each joint comprising an independent actuator,each actuator comprising an independently controllable motor. Eachindependently controllable joint represents an independent degree offreedom available to the robotic arm. Each of the arms 12 have sevenjoints, and thus provide seven degrees of freedom. A multitude of jointsresult in a multitude of degrees of freedom, allowing for “redundant”degrees of freedom. Redundant degrees of freedom allow the robotic arms12 to position their respective end effectors 22 at a specific position,orientation, and trajectory in space using different linkage positionsand joint angles. This allows for the system to position and direct amedical instrument from a desired point in space while allowing thephysician to move the arm joints into a clinically advantageous positionaway from the patient to create greater access, while avoiding armcollisions.

The cart base 15 balances the weight of the column 14, carriage 17, andarms 12 over the floor. Accordingly, the cart base 15 houses heaviercomponents, such as electronics, motors, power supply, as well ascomponents that either enable movement and/or immobilize the cart. Forexample, the cart base 15 includes rollable wheel-shaped casters 25 thatallow for the cart to easily move around the room prior to a procedure.After reaching the appropriate position, the casters 25 may beimmobilized using wheel locks to hold the cart 11 in place during theprocedure.

Positioned at the vertical end of column 14, the console 16 allows forboth a user interface for receiving user input and a display screen (ora dual-purpose device such as, for example, a touchscreen 26) to providethe physician user with both pre-operative and intra-operative data.Potential pre-operative data on the touchscreen 26 may includepre-operative plans, navigation and mapping data derived frompre-operative computerized tomography (CT) scans, and/or notes frompre-operative patient interviews. Intra-operative data on display mayinclude optical information provided from the tool, sensor andcoordinate information from sensors, as well as vital patientstatistics, such as respiration, heart rate, and/or pulse. The console16 may be positioned and tilted to allow a physician to access theconsole from the side of the column 14 opposite carriage 17. From thisposition, the physician may view the console 16, robotic arms 12, andpatient while operating the console 16 from behind the cart 11. Asshown, the console 16 also includes a handle 27 to assist withmaneuvering and stabilizing cart 11.

FIG. 3 illustrates an embodiment of a robotically-enabled system 10arranged for ureteroscopy. In a ureteroscopic procedure, the cart 11 maybe positioned to deliver a ureteroscope 32, a procedure-specificendoscope designed to traverse a patient's urethra and ureter, to thelower abdominal area of the patient. In a ureteroscopy, it may bedesirable for the ureteroscope 32 to be directly aligned with thepatient's urethra to reduce friction and forces on the sensitive anatomyin the area. As shown, the cart 11 may be aligned at the foot of thetable to allow the robotic arms 12 to position the ureteroscope 32 fordirect linear access to the patient's urethra. From the foot of thetable, the robotic arms 12 may insert the ureteroscope 32 along thevirtual rail 33 directly into the patient's lower abdomen through theurethra.

After insertion into the urethra, using similar control techniques as inbronchoscopy, the ureteroscope 32 may be navigated into the bladder,ureters, and/or kidneys for diagnostic and/or therapeutic applications.For example, the ureteroscope 32 may be directed into the ureter andkidneys to break up kidney stone build up using laser or ultrasoniclithotripsy device deployed down the working channel of the ureteroscope32. After lithotripsy is complete, the resulting stone fragments may beremoved using baskets deployed down the ureteroscope 32.

FIG. 4 illustrates an embodiment of a robotically-enabled systemsimilarly arranged for a vascular procedure. In a vascular procedure,the system 10 may be configured such the cart 11 may deliver a medicalinstrument 34, such as a steerable catheter, to an access point in thefemoral artery in the patient's leg. The femoral artery presents both alarger diameter for navigation as well as relatively less circuitous andtortuous path to the patient's heart, which simplifies navigation. As ina ureteroscopic procedure, the cart 11 may be positioned towards thepatient's legs and lower abdomen to allow the robotic arms 12 to providea virtual rail 35 with direct linear access to the femoral artery accesspoint in the patient's thigh/hip region. After insertion into theartery, the medical instrument 34 may be directed and inserted bytranslating the instrument drivers 28. Alternatively, the cart may bepositioned around the patient's upper abdomen in order to reachalternative vascular access points, such as, for example, the carotidand brachial arteries near the shoulder and wrist.

B. Robotic System—Table

Embodiments of the robotically-enabled medical system may alsoincorporate the patient's table. Incorporation of the table reduces theamount of capital equipment within the operating room by removing thecart, which allows greater access to the patient. FIG. 5 illustrates anembodiment of such a robotically-enabled system arranged for abronchoscopy procedure. System 36 includes a support structure or column37 for supporting platform 38 (shown as a “table” or “bed”) over thefloor. Much like in the cart-based systems, the end effectors of therobotic arms 39 of the system 36 comprise instrument drivers 42 that aredesigned to manipulate an elongated medical instrument, such as abronchoscope 40 in FIG. 5, through or along a virtual rail 41 formedfrom the linear alignment of the instrument drivers 42. In practice, aC-arm for providing fluoroscopic imaging may be positioned over thepatient's upper abdominal area by placing the emitter and detectoraround table 38.

FIG. 6 provides an alternative view of the system 36 without the patientand medical instrument for discussion purposes. As shown, the column 37may include one or more carriages 43 shown as ring-shaped in the system36, from which the one or more robotic arms 39 may be based. Thecarriages 43 may translate along a vertical column interface 44 thatruns the length of the column 37 to provide different vantage pointsfrom which the robotic arms 39 may be positioned to reach the patient.The carriage(s) 43 may rotate around the column 37 using a mechanicalmotor positioned within the column 37 to allow the robotic arms 39 tohave access to multiples sides of the table 38, such as, for example,both sides of the patient. In embodiments with multiple carriages, thecarriages may be individually positioned on the column and may translateand/or rotate independent of the other carriages. While carriages 43need not surround the column 37 or even be circular, the ring-shape asshown facilitates rotation of the carriages 43 around the column 37while maintaining structural balance. Rotation and translation of thecarriages 43 allows the system to align the medical instruments, such asendoscopes and laparoscopes, into different access points on thepatient. In other embodiments (not shown), the system 36 can include apatient table or bed with adjustable arm supports in the form of bars orrails extending alongside it. One or more robotic arms 39 (e.g., via ashoulder with an elbow joint) can be attached to the adjustable armsupports, which can be vertically adjusted. By providing verticaladjustment, the robotic arms 39 are advantageously capable of beingstowed compactly beneath the patient table or bed, and subsequentlyraised during a procedure.

The arms 39 may be mounted on the carriages through a set of arm mounts45 comprising a series of joints that may individually rotate and/ortelescopically extend to provide additional configurability to therobotic arms 39. Additionally, the arm mounts 45 may be positioned onthe carriages 43 such that, when the carriages 43 are appropriatelyrotated, the arm mounts 45 may be positioned on either the same side oftable 38 (as shown in FIG. 6), on opposite sides of table 38 (as shownin FIG. 9), or on adjacent sides of the table 38 (not shown).

The column 37 structurally provides support for the table 38, and a pathfor vertical translation of the carriages. Internally, the column 37 maybe equipped with lead screws for guiding vertical translation of thecarriages, and motors to mechanize the translation of said carriagesbased the lead screws. The column 37 may also convey power and controlsignals to the carriage 43 and robotic arms 39 mounted thereon.

The table base 46 serves a similar function as the cart base 15 in cart11 shown in FIG. 2, housing heavier components to balance the table/bed38, the column 37, the carriages 43, and the robotic arms 39. The tablebase 46 may also incorporate rigid casters to provide stability duringprocedures. Deployed from the bottom of the table base 46, the castersmay extend in opposite directions on both sides of the base 46 andretract when the system 36 needs to be moved.

Continuing with FIG. 6, the system 36 may also include a tower (notshown) that divides the functionality of system 36 between table andtower to reduce the form factor and bulk of the table. As in earlierdisclosed embodiments, the tower may be provide a variety of supportfunctionalities to table, such as processing, computing, and controlcapabilities, power, fluidics, and/or optical and sensor processing. Thetower may also movable to be positioned away from the patient to improvephysician access and de-clutter the operating room. Additionally,placing components in the tower allows for more storage space in thetable base for potential stowage of the robotic arms. The tower may alsoinclude a master controller or console that provides both a userinterface for user input, such as keyboard and/or pendant, as well as adisplay screen (or touchscreen) for pre-operative and intra-operativeinformation, such as real-time imaging, navigation, and trackinginformation. In some embodiments, the tower may also contain holders forgas tanks to be used for insufflation.

In some embodiments, a table base may stow and store the robotic armswhen not in use. FIG. 7 illustrates a system 47 that stows robotic armsin an embodiment of the table-based system. In system 47, carriages 48may be vertically translated into base 49 to stow robotic arms 50, armmounts 51, and the carriages 48 within the base 49. Base covers 52 maybe translated and retracted open to deploy the carriages 48, arm mounts51, and arms 50 around column 53, and closed to stow to protect themwhen not in use. The base covers 52 may be sealed with a membrane 54along the edges of its opening to prevent dirt and fluid ingress whenclosed.

FIG. 8 illustrates an embodiment of a robotically-enabled table-basedsystem configured for a ureteroscopy procedure. In a ureteroscopy, thetable 38 may include a swivel portion 55 for positioning a patientoff-angle from the column 37 and table base 46. The swivel portion 55may rotate or pivot around a pivot point (e.g., located below thepatient's head) in order to position the bottom portion of the swivelportion 55 away from the column 37. For example, the pivoting of theswivel portion 55 allows a C-arm (not shown) to be positioned over thepatient's lower abdomen without competing for space with the column (notshown) below table 38. By rotating the carriage 35 (not shown) aroundthe column 37, the robotic arms 39 may directly insert a ureteroscope 56along a virtual rail 57 into the patient's groin area to reach theurethra. In a ureteroscopy, stirrups 58 may also be fixed to the swivelportion 55 of the table 38 to support the position of the patient's legsduring the procedure and allow clear access to the patient's groin area.

In a laparoscopic procedure, through small incision(s) in the patient'sabdominal wall, minimally invasive instruments may be inserted into thepatient's anatomy. In some embodiments, the minimally invasiveinstruments comprise an elongated rigid member, such as a shaft, whichis used to access anatomy within the patient. After inflation of thepatient's abdominal cavity, the instruments, often referred to aslaparoscopes, may be directed to perform surgical tasks, such asgrasping, cutting, ablating, suturing, etc. FIG. 9 illustrates anembodiment of a robotically-enabled table-based system configured for alaparoscopic procedure. As shown in FIG. 9, the carriages 43 of thesystem 36 may be rotated and vertically adjusted to position pairs ofthe robotic arms 39 on opposite sides of the table 38, such thatinstruments (e.g., laparoscopes) 59 may be positioned using the armmounts 45 to be passed through minimal incisions on both sides of thepatient to reach his/her abdominal cavity.

To accommodate laparoscopic procedures, the robotically-enabled tablesystem may also tilt the platform to a desired angle. FIG. 10illustrates an embodiment of the robotically-enabled medical system withpitch or tilt adjustment. As shown in FIG. 10, the system 36 mayaccommodate tilt of the table 38 to position one portion of the table ata greater distance from the floor than the other. Additionally, the armmounts 45 may rotate to match the tilt such that the arms 39 maintainthe same planar relationship with table 38. To accommodate steeperangles, the column 37 may also include telescoping portions 60 thatallow vertical extension of column 37 to keep the table 38 from touchingthe floor or colliding with base 46.

FIG. 11 provides a detailed illustration of the interface between thetable 38 and the column 37. Pitch rotation mechanism 61 may beconfigured to alter the pitch angle of the table 38 relative to thecolumn 37 in multiple degrees of freedom. The pitch rotation mechanism61 may be enabled by the positioning of orthogonal axes 1, 2 at thecolumn-table interface, each axis actuated by a separate motor 3, 4responsive to an electrical pitch angle command. Rotation along onescrew 5 would enable tilt adjustments in one axis 1, while rotationalong the other screw 6 would enable tilt adjustments along the otheraxis 2. In some embodiments, a ball joint can be used to alter the pitchangle of the table 38 relative to the column 37 in multiple degrees offreedom.

For example, pitch adjustments are particularly useful when trying toposition the table in a Trendelenburg position, i.e., position thepatient's lower abdomen at a higher position from the floor than thepatient's lower abdomen, for lower abdominal surgery. The Trendelenburgposition causes the patient's internal organs to slide towards his/herupper abdomen through the force of gravity, clearing out the abdominalcavity for minimally invasive tools to enter and perform lower abdominalsurgical procedures, such as laparoscopic prostatectomy.

C. Instrument Driver & Interface

The end effectors of the system's robotic arms comprise (i) aninstrument driver (alternatively referred to as “instrument drivemechanism” or “instrument device manipulator” (IDM)) that incorporateelectro-mechanical means for actuating the medical instrument and (ii) aremovable or detachable medical instrument which may be devoid of anyelectro-mechanical components, such as motors. This dichotomy may bedriven by the need to sterilize medical instruments used in medicalprocedures, and the inability to adequately sterilize expensive capitalequipment due to their intricate mechanical assemblies and sensitiveelectronics. Accordingly, the medical instruments may be designed to bedetached, removed, and interchanged from the instrument driver (and thusthe system) for individual sterilization or disposal by the physician orthe physician's staff. In contrast, the instrument drivers need not bechanged or sterilized, and may be draped for protection.

FIG. 12 illustrates an example instrument driver. Positioned at thedistal end of a robotic arm, instrument driver 62 comprises of one ormore drive units 63 arranged with parallel axes to provide controlledtorque to a medical instrument via drive shafts 64. Each drive unit 63comprises an individual drive shaft 64 for interacting with theinstrument, a gear head 65 for converting the motor shaft rotation to adesired torque, a motor 66 for generating the drive torque, an encoder67 to measure the speed of the motor shaft and provide feedback to thecontrol circuitry, and control circuity 68 for receiving control signalsand actuating the drive unit. Each drive unit 63 being independentcontrolled and motorized, the instrument driver 62 may provide multiple(four as shown in FIG. 12) independent drive outputs to the medicalinstrument. In operation, the control circuitry 68 would receive acontrol signal, transmit a motor signal to the motor 66, compare theresulting motor speed as measured by the encoder 67 with the desiredspeed, and modulate the motor signal to generate the desired torque.

For procedures that require a sterile environment, the robotic systemmay incorporate a drive interface, such as a sterile adapter connectedto a sterile drape, that sits between the instrument driver and themedical instrument. The chief purpose of the sterile adapter is totransfer angular motion from the drive shafts of the instrument driverto the drive inputs of the instrument while maintaining physicalseparation, and thus sterility, between the drive shafts and driveinputs. Accordingly, an example sterile adapter may comprise of a seriesof rotational inputs and outputs intended to be mated with the driveshafts of the instrument driver and drive inputs on the instrument.Connected to the sterile adapter, the sterile drape, comprised of athin, flexible material such as transparent or translucent plastic, isdesigned to cover the capital equipment, such as the instrument driver,robotic arm, and cart (in a cart-based system) or table (in atable-based system). Use of the drape would allow the capital equipmentto be positioned proximate to the patient while still being located inan area not requiring sterilization (i.e., non-sterile field). On theother side of the sterile drape, the medical instrument may interfacewith the patient in an area requiring sterilization (i.e., sterilefield).

D. Medical Instrument

FIG. 13 illustrates an example medical instrument with a pairedinstrument driver. Like other instruments designed for use with arobotic system, medical instrument 70 comprises an elongated shaft 71(or elongate body) and an instrument base 72. The instrument base 72,also referred to as an “instrument handle” due to its intended designfor manual interaction by the physician, may generally compriserotatable drive inputs 73, e.g., receptacles, pulleys or spools, thatare designed to be mated with drive outputs 74 that extend through adrive interface on instrument driver 75 at the distal end of robotic arm76. When physically connected, latched, and/or coupled, the mated driveinputs 73 of instrument base 72 may share axes of rotation with thedrive outputs 74 in the instrument driver 75 to allow the transfer oftorque from drive outputs 74 to drive inputs 73. In some embodiments,the drive outputs 74 may comprise splines that are designed to mate withreceptacles on the drive inputs 73.

The elongated shaft 71 is designed to be delivered through either ananatomical opening or lumen, e.g., as in endoscopy, or a minimallyinvasive incision, e.g., as in laparoscopy. The elongated shaft 66 maybe either flexible (e.g., having properties similar to an endoscope) orrigid (e.g., having properties similar to a laparoscope) or contain acustomized combination of both flexible and rigid portions. Whendesigned for laparoscopy, the distal end of a rigid elongated shaft maybe connected to an end effector extending from a jointed wrist formedfrom a clevis with at least one degree of freedom and a surgical tool ormedical instrument, such as, for example, a grasper or scissors, thatmay be actuated based on force from the tendons as the drive inputsrotate in response to torque received from the drive outputs 74 of theinstrument driver 75. When designed for endoscopy, the distal end of aflexible elongated shaft may include a steerable or controllable bendingsection that may be articulated and bent based on torque received fromthe drive outputs 74 of the instrument driver 75.

Torque from the instrument driver 75 is transmitted down the elongatedshaft 71 using tendons along the shaft 71. These individual tendons,such as pull wires, may be individually anchored to individual driveinputs 73 within the instrument handle 72. From the handle 72, thetendons are directed down one or more pull lumens along the elongatedshaft 71 and anchored at the distal portion of the elongated shaft 71,or in the wrist at a distal portion of the elongated shaft. During aprocedure, such as a laparoscopic, endoscopic, or hybrid procedure,these tendons may be coupled to a distally mounted end effector, such asa wrist, grasper, or scissor. Under such an arrangement, torque exertedon drive inputs 73 would transfer tension to the tendon, thereby causingthe end effector to actuate in some way. In some embodiments, during asurgical procedure, the tendon may cause a joint to rotate about anaxis, thereby causing the end effector to move in one direction oranother. Alternatively, the tendon may be connected to one or more jawsof a grasper at distal end of the elongated shaft 71, where tension fromthe tendon cause the grasper to close.

In endoscopy, the tendons may be coupled to a bending or articulatingsection positioned along the elongated shaft 71 (e.g., at the distalend) via adhesive, control ring, or other mechanical fixation. Whenfixedly attached to the distal end of a bending section, torque exertedon drive inputs 73 would be transmitted down the tendons, causing thesofter, bending section (sometimes referred to as the articulablesection or region) to bend or articulate. Along the non-bendingsections, it may be advantageous to spiral or helix the individual pulllumens that direct the individual tendons along (or inside) the walls ofthe endoscope shaft to balance the radial forces that result fromtension in the pull wires. The angle of the spiraling and/or spacingthere between may be altered or engineered for specific purposes,wherein tighter spiraling exhibits lesser shaft compression under loadforces, while lower amounts of spiraling results in greater shaftcompression under load forces, but also exhibits limits bending. On theother end of the spectrum, the pull lumens may be directed parallel tothe longitudinal axis of the elongated shaft 71 to allow for controlledarticulation in the desired bending or articulable sections.

In endoscopy, the elongated shaft 71 houses a number of components toassist with the robotic procedure. The shaft may comprise of a workingchannel for deploying surgical tools (or medical instruments),irrigation, and/or aspiration to the operative region at the distal endof the shaft 71. The shaft 71 may also accommodate wires and/or opticalfibers to transfer signals to/from an optical assembly at the distaltip, which may include of an optical camera. The shaft 71 may alsoaccommodate optical fibers to carry light from proximally-located lightsources, such as light emitting diodes, to the distal end of the shaft.

At the distal end of the instrument 70, the distal tip may also comprisethe opening of a working channel for delivering tools for diagnosticand/or therapy, irrigation, and aspiration to an operative site. Thedistal tip may also include a port for a camera, such as a fiberscope ora digital camera, to capture images of an internal anatomical space.Relatedly, the distal tip may also include ports for light sources forilluminating the anatomical space when using the camera.

In the example of FIG. 13, the drive shaft axes, and thus the driveinput axes, are orthogonal to the axis of the elongated shaft. Thisarrangement, however, complicates roll capabilities for the elongatedshaft 71. Rolling the elongated shaft 71 along its axis while keepingthe drive inputs 73 static results in undesirable tangling of thetendons as they extend off the drive inputs 73 and enter pull lumenswithin the elongated shaft 71. The resulting entanglement of suchtendons may disrupt any control algorithms intended to predict movementof the flexible elongated shaft 71 during an endoscopic procedure.

FIG. 14 illustrates an alternative design for an instrument driver andinstrument where the axes of the drive units are parallel to the axis ofthe elongated shaft of the instrument. As shown, a circular instrumentdriver 80 comprises four drive units with their drive outputs 81 alignedin parallel at the end of a robotic arm 82. The drive units, and theirrespective drive outputs 81, are housed in a rotational assembly 83 ofthe instrument driver 80 that is driven by one of the drive units withinthe assembly 83. In response to torque provided by the rotational driveunit, the rotational assembly 83 rotates along a circular bearing thatconnects the rotational assembly 83 to the non-rotational portion 84 ofthe instrument driver. Power and controls signals may be communicatedfrom the non-rotational portion 84 of the instrument driver 80 to therotational assembly 83 through electrical contacts may be maintainedthrough rotation by a brushed slip ring connection (not shown). In otherembodiments, the rotational assembly 83 may be responsive to a separatedrive unit that is integrated into the non-rotatable portion 84, andthus not in parallel to the other drive units. The rotational mechanism83 allows the instrument driver 80 to rotate the drive units, and theirrespective drive outputs 81, as a single unit around an instrumentdriver axis 85.

Like earlier disclosed embodiments, an instrument 86 may comprise of anelongated shaft portion 88 and an instrument base 87 (shown with atransparent external skin for discussion purposes) comprising aplurality of drive inputs 89 (such as receptacles, pulleys, and spools)that are configured to receive the drive outputs 81 in the instrumentdriver 80. Unlike prior disclosed embodiments, instrument shaft 88extends from the center of instrument base 87 with an axis substantiallyparallel to the axes of the drive inputs 89, rather than orthogonal asin the design of FIG. 13.

When coupled to the rotational assembly 83 of the instrument driver 80,the medical instrument 86, comprising instrument base 87 and instrumentshaft 88, rotates in combination with the rotational assembly 83 aboutthe instrument driver axis 85. Since the instrument shaft 88 ispositioned at the center of instrument base 87, the instrument shaft 88is coaxial with instrument driver axis 85 when attached. Thus, rotationof the rotational assembly 83 causes the instrument shaft 88 to rotateabout its own longitudinal axis. Moreover, as the instrument base 87rotates with the instrument shaft 88, any tendons connected to the driveinputs 89 in the instrument base 87 are not tangled during rotation.Accordingly, the parallelism of the axes of the drive outputs 81, driveinputs 89, and instrument shaft 88 allows for the shaft rotation withouttangling any control tendons.

E. Navigation and Control

Traditional endoscopy may involve the use of fluoroscopy (e.g., as maybe delivered through a C-arm) and other forms of radiation-based imagingmodalities to provide endoluminal guidance to an operator physician. Incontrast, the robotic systems contemplated by this disclosure canprovide for non-radiation-based navigational and localization means toreduce physician exposure to radiation and reduce the amount ofequipment within the operating room. As used herein, the term“localization” may refer to determining and/or monitoring the positionof objects in a reference coordinate system. Technologies such aspre-operative mapping, computer vision, real-time EM tracking, and robotcommand data may be used individually or in combination to achieve aradiation-free operating environment. In other cases, whereradiation-based imaging modalities are still used, the pre-operativemapping, computer vision, real-time EM tracking, and robot command datamay be used individually or in combination to improve upon theinformation obtained solely through radiation-based imaging modalities.

FIG. 15 is a block diagram illustrating a localization system 90 thatestimates a location of one or more elements of the robotic system, suchas the location of the instrument, in accordance to an exampleembodiment. The localization system 90 may be a set of one or morecomputer devices configured to execute one or more instructions. Thecomputer devices may be embodied by a processor (or processors) andcomputer-readable memory in one or more components discussed above. Byway of example and not limitation, the computer devices may be in thetower 30 shown in FIG. 1, the cart shown in FIGS. 1-4, the beds shown inFIGS. 5-10, etc.

As shown in FIG. 15, the localization system 90 may include alocalization module 95 that processes input data 91-94 to generatelocation data 96 for the distal tip of a medical instrument. Thelocation data 96 may be data or logic that represents a location and/ororientation of the distal end of the instrument relative to a frame ofreference. The frame of reference can be a frame of reference relativeto the anatomy of the patient or to a known object, such as an EM fieldgenerator (see discussion below for the EM field generator).

The various input data 91-94 are now described in greater detail.Pre-operative mapping may be accomplished through the use of thecollection of low dose CT scans. Pre-operative CT scans arereconstructed into three-dimensional images, which are visualized, e.g.,as “slices” of a cutaway view of the patient's internal anatomy. Whenanalyzed in the aggregate, image-based models for anatomical cavities,spaces and structures of the patient's anatomy, such as a patient lungnetwork, may be generated. Techniques such as center-line geometry maybe determined and approximated from the CT images to develop athree-dimensional volume of the patient's anatomy, referred to as modeldata 91 (also referred to as “preoperative model data” when generatedusing only preoperative CT scans). The use of center-line geometry isdiscussed in U.S. patent application Ser. No. 14/523,760, the contentsof which are herein incorporated in its entirety. Network topologicalmodels may also be derived from the CT-images, and are particularlyappropriate for bronchoscopy.

In some embodiments, the instrument may be equipped with a camera toprovide vision data 92. The localization module 95 may process thevision data to enable one or more vision-based location tracking. Forexample, the preoperative model data may be used in conjunction with thevision data 92 to enable computer vision-based tracking of the medicalinstrument (e.g., an endoscope or an instrument advance through aworking channel of the endoscope). For example, using the preoperativemodel data 91, the robotic system may generate a library of expectedendoscopic images from the model based on the expected path of travel ofthe endoscope, each image linked to a location within the model.Intra-operatively, this library may be referenced by the robotic systemin order to compare real-time images captured at the camera (e.g., acamera at a distal end of the endoscope) to those in the image libraryto assist localization.

Other computer vision-based tracking techniques use feature tracking todetermine motion of the camera, and thus the endoscope. Some features ofthe localization module 95 may identify circular geometries in thepreoperative model data 91 that correspond to anatomical lumens andtrack the change of those geometries to determine which anatomical lumenwas selected, as well as the relative rotational and/or translationalmotion of the camera. Use of a topological map may further enhancevision-based algorithms or techniques.

Optical flow, another computer vision-based technique, may analyze thedisplacement and translation of image pixels in a video sequence in thevision data 92 to infer camera movement. Examples of optical flowtechniques may include motion detection, object segmentationcalculations, luminance, motion compensated encoding, stereo disparitymeasurement, etc. Through the comparison of multiple frames overmultiple iterations, movement and location of the camera (and thus theendoscope) may be determined.

The localization module 95 may use real-time EM tracking to generate areal-time location of the endoscope in a global coordinate system thatmay be registered to the patient's anatomy, represented by thepreoperative model. In EM tracking, an EM sensor (or tracker) comprisingof one or more sensor coils embedded in one or more locations andorientations in a medical instrument (e.g., an endoscopic tool) measuresthe variation in the EM field created by one or more static EM fieldgenerators positioned at a known location. The location informationdetected by the EM sensors is stored as EM data 93. The EM fieldgenerator (or transmitter), may be placed close to the patient to createa low intensity magnetic field that the embedded sensor may detect. Themagnetic field induces small currents in the sensor coils of the EMsensor, which may be analyzed to determine the distance and anglebetween the EM sensor and the EM field generator. These distances andorientations may be intra-operatively “registered” to the patientanatomy (e.g., the preoperative model) in order to determine thegeometric transformation that aligns a single location in the coordinatesystem with a position in the pre-operative model of the patient'sanatomy. Once registered, an embedded EM tracker in one or morepositions of the medical instrument (e.g., the distal tip of anendoscope) may provide real-time indications of the progression of themedical instrument through the patient's anatomy.

Robotic command and kinematics data 94 may also be used by thelocalization module 95 to provide localization data 96 for the roboticsystem. Device pitch and yaw resulting from articulation commands may bedetermined during pre-operative calibration. Intra-operatively, thesecalibration measurements may be used in combination with known insertiondepth information to estimate the position of the instrument.Alternatively, these calculations may be analyzed in combination withEM, vision, and/or topological modeling to estimate the position of themedical instrument within the network.

As FIG. 15 shows, a number of other input data can be used by thelocalization module 95. For example, although not shown in FIG. 15, aninstrument utilizing shape-sensing fiber can provide shape data that thelocalization module 95 can use to determine the location and shape ofthe instrument.

The localization module 95 may use the input data 91-94 incombination(s). In some cases, such a combination may use aprobabilistic approach where the localization module 95 assigns aconfidence weight to the location determined from each of the input data91-94. Thus, where the EM data may not be reliable (as may be the casewhere there is EM interference) the confidence of the locationdetermined by the EM data 93 can be decrease and the localization module95 may rely more heavily on the vision data 92 and/or the roboticcommand and kinematics data 94.

As discussed above, the robotic systems discussed herein may be designedto incorporate a combination of one or more of the technologies above.The robotic system's computer-based control system, based in the tower,bed and/or cart, may store computer program instructions, for example,within a non-transitory computer-readable storage medium such as apersistent magnetic storage drive, solid state drive, or the like, that,upon execution, cause the system to receive and analyze sensor data anduser commands, generate control signals throughout the system, anddisplay the navigational and localization data, such as the position ofthe instrument within the global coordinate system, anatomical map, etc.

2. Introduction to Pre-Procedure Robotic Arm Setup

Embodiments of the disclosure relate to systems and techniques for thepositioning of one or more robotic arms prior to performing a medicalprocedure. Depending on the type of medical procedure, there may belimitations to the extent to which a robotic arm may be positioned ormoved during setup of a surgical system. For example, certain physicalor mechanical considerations, such as the shape and dimensions (e.g.,length) of the medical instrument, the shape and physiologicalcharacteristics of the patient's luminal network, the location of atarget destination for the procedure, the working area of the roboticarm(s), etc., may limit the area or volume in which the robotic arm(s)may be positioned during setup.

Setup procedures for surgical robotic systems may aid in ensuring and/orverifying that a given procedure is achievable. For example, a targetdestination or operative site may be located at a certain distance froman access point in the patient's body. Further, the length of themedical instrument and the stroke length of the robotic arm may besubstantially fixed for a given surgical robotic system (e.g., roboticarm stroke length and medical instrument length may be standardized). Asused herein, the stroke length of a robotic arm generally refers to theability or extent to which the robotic arm can insert an instrumenttowards a target, such as, for example, from a startingposition/location of a reference point of the robotic arm to an endposition/location of the reference point. In one embodiment, thereference point may be the IDM of the robotic arm, and the stroke lengthmay refer to the distance from a position of the IDM in the initial poseof the robotic arm for beginning the medical procedure (e.g., theinitial pose and position of the robotic arm that facilitates theloading/attachment of instruments to the robotic arm, referred to hereinas the load instruments pose) to a position of the IDM during maximuminsertion of a medical instrument. For example, the stroke length of arobotic arm may determine whether the robotic arm is able to reach atarget site/area from a given access point on the patient's body.Depending on the context, the “stroke length of the robotic arm” mayalso be used interchangeably with the “stroke length of a medicalinstrument” since the movement of the robotic arm may be directlycorrelated with the insertion/retraction of the medical instrument.

Given the length of the medical instrument, the distance from the accesspoint to the target destination, and the maximum achievable strokelength of the robotic arm, there is a limit to the locations that may beused as the initial pose of the robotic arm prior to the surgicalprocedure. That is, certain initial poses of the robotic arm(s) willallow the medical instrument to reach the target destination while otherinitial poses may not enable the target destination to be reached. Theplacement of the cart with respect to the patient's access point, andthus, the location of the area/volume in which the robotic arm(s) can befreely positioned, may affect the achievable stroke length of therobotic arm(s). If the achievable stroke length is reduced to a lengththat is less than the distance from the access point to the targetdestination for a given medical procedure, the medical procedure may notbe able to be performed based on the setup of the robotic system.

Aspects of this disclosure may relate to systems and techniques that aida user (e.g., a technician or surgeon) in determining whether thedesired procedure can be completed based on a given initial pose of therobotic arm. Such techniques may solve the problems associated withbeing unable to reach a target destination during a procedure.

A. Bronchoscopy System Example

Aspects of this disclosure will be generally described usingbronchoscopy as an exemplary medical procedure. However, this disclosureis also applicable to other types of medical procedures performed by asurgical robotic system, such as, for example, ureteroscopy,gastroenterology, etc.

FIG. 16 illustrates an embodiment of a robotic system arranged fordiagnostic and/or therapeutic bronchoscopy procedure(s) in accordancewith aspects of this disclosure. As shown in FIG. 16, the system 100 mayinclude a cart 111, one or more robotic arm(s) 112, a medicalinstrument, such as a steerable endoscope 113, and a patient introducer115. The cart 111 may include a processor (not illustrated), a memory(not illustrated), and a display 120 configured to display informationrelated to the positioning of the robotic arm(s) 112. However, dependingon the embodiment, one or more of the processor, the memory, and thedisplay may be located on or within another device, such as, e.g., onthe moveable tower 30 illustrated in FIG. 1. Additionally, in otherimplementations, a feedback device other than the display 120 may beused in place of, or in addition to, the display 120. Other feedbackdevices which may be employed include haptic devices, speakers,force-feedback actuated via one or more of the robotic arm(s) 112, oneor more light-emitting diode(s), etc.

The robotic arm(s) 112 may be configured to manipulate the medicalinstrument (e.g., the steerable endoscope 113) prior to and/or duringthe procedure. During an arm setup phase (e.g., prior to beginning themedical procedure), the position of the robotic arm(s) 112 may beadjustable by the user. In particular, it may be important for at leastone of the robotic arm(s) to be aligned with the patient. This alignmentmay enable the system to track the entry/access point of the medicalinstrument into the patient. Depending on the implementation, the systemmay be configured to allow the user to directly move the arm(s) 112 byapplying a force directly to a portion of the robotic arm 112. Forexample, the system may be configured to detect when a user grabs one ofthe robotic arm(s) 112 and physically moves the arm 112 into a desiredposition by applying a force (e.g., pushing or pulling on the arm) tomove the arm 112. In certain implementations, the system may acceptinput from a user to toggle the arm into and out of an admittance modein which the arm accepts force as input for movement of the arm. Inother implementations, the system may include an input device configuredto enable the user to adjust the position of one or more of the roboticarm(s) via the input device for the control of the robotic arms 112.

As will be described in greater detail below, the memory may beconfigured to store instructions, that when executed, cause theprocessor to perform one or more techniques in accordance with aspectsof this disclosure. The memory may be further configured to store datarelevant to the pre-procedure robotic arm setup. For example, the memorymay store a mapping of an anatomy of a patient. The mapping may comprisedata regarding (i) a target region within the anatomy and (ii) a pathfrom an access point of the patient to the target region. The mappingmay comprise or be based on procedure data related to the medicalprocedure being performed. The procedure data may include dataregarding: the type of procedure being performed, the type of instrumentinvolved in the procedure (which may relate to the type of theprocedure), attributes of the instrument (e.g., the length of theinstrument, the number of IDMs required for manipulating the instrument,etc.), the patient's anatomy relevant to the procedure (e.g., thelocation of a target region within the anatomy, a path from an accesspoint of the patient to the target region, physiological characteristicsor dimensions of the patient's anatomy, etc.), the arm setup (which mayalso relate to the type of the procedure, the type of the instrument,and the attributes of the instrument), etc. For example, the mapping mayinclude the location of the target region within the anatomy and thepath from the access point of the patient to the target region, whichmay be determined based on the procedure data.

B. Robotic Arm Setup

Positioning of one or more of the robotic arm(s) 112 may be one part ofa setup procedure for preparing the robotic arm system for a medicalprocedure. The specific setup procedure used may depend on the medicalprocedure being performed, the configuration of the robotic system(e.g., whether the arms are attached to a cart (see FIG. 16) or attachedto a column supporting the platform (see FIG. 6)), etc.

FIG. 17A is a flow-chart which illustrates features of an example setupprocedure for a medical procedure (e.g., bronchoscopy) in accordancewith aspects of this disclosure. The method 1700 illustrated in FIG. 17Ais merely an example implementation and the method 1700 may be modifiedby adding, removing, and or modifying one or more of the blocksassociated with the method 1700.

The method 1700 begins at block 1701. At block 1705, the method 1700involves moving the cart to an initial position. For example, a user maymove the cart to be positioned proximate to (e.g., within a defineddistance of) the patient's access point. Once the cart has been movedinto position, the user may immobilize the cart by, for example, lockingthe casters of the cart. It is to be appreciated that not all roboticsystems may utilize a cart and this block is optional for those systemsthat do utilize a cart.

Block 1710 may involve an arm setup phase, in which the user may placeone or more of the robotic arm(s) into an initial pose where the roboticarm(s) are aligned with the patient prior to performing the procedure.Thus, the arm setup phase may include an alignment step for aligning oneor more of the robotic arm(s) 112 with an access point of the patient.Since the access point used may depend on the type of the medicalprocedure being performed, the specific alignment procedure may dependon the medical procedure type. In the bronchoscopy example, a patientintroducer (a device which guides the bronchoscope into the patient'smouth) may be installed into the patient's mouth. In one implementationof a bronchoscopy setup procedure, the user may align a first one of therobotic arms with the patient introducer. The remaining robotic arm(s)may, e.g., automatically align with the pose of the first robotic armselected by the user. As described above, the user may be able todirectly move the robotic arm by pressing an admittance button whichallows the user to direct movement of the robotic arm by applyingforce(s) thereto. In other implementations, the first robotic arm maytrack the patient introducer via one or more position tracking devices,enabling the first robotic arm to be automatically aligned with thepatient introducer by the system.

The system may provide an indication of a boundary for an initial poseof the first robotic arm to the user. In certain embodiments, the systemmay, during an arm setup phase prior to performing a medical procedure,provide the indication of the boundary during movement of the firstrobotic arm. The boundary may be set by the system to ensure that thepose of the first robotic arm does not interfere with the medicalprocedure. In one implementation, the boundary may be set as an area orvolume in which the first robotic arm can be freely positioned withoutreducing the stroke length of the robotic arm by more than a thresholdamount. In a bronchoscopy example, the boundary may define an area inwhich the robotic arm may be aligned with the patient introducer suchthat the distance between the initial pose of the robotic arm (e.g., thepose of the robotic arm in alignment with the patient introducer) andthe load instruments pose is equal to or greater than a threshold strokelength. In certain embodiments, the threshold stroke length may beselected such that a target region associated with the medical procedurecan be reached when the stroke length achievable by the robotic arm isgreater than the threshold stroke length.

After the first robotic arm is aligned with the patient introducer, atblock 1715, the method involves retracting the robotic arm(s) into aload instrument(s) pose. In some embodiments, the system may retract therobotic arm(s) into the load instrument(s) pose in response to receivinga load-instrument-pose input or command e.g., from the user. This inputmay indicate that the alignment step has been completed and thatinstrument(s) (e.g., the sheath and leader of the bronchoscope) are tobe loaded onto the robotic arm(s). At block 1720, the method involvesloading the medical instrument(s) onto the corresponding robotic arm(s).The method 1700 ends at block 1725.

FIG. 17B is a flowchart which illustrates features of another examplesetup procedure for a medical procedure in accordance with aspects ofthis disclosure. The flowchart of FIG. 17B illustrates an example methodoperable by a surgical robotic system, or component(s) thereof, forpositioning one or more robotic arms prior to performing a medicalprocedure in accordance with aspects of this disclosure. For example,the steps of method 1750 illustrated in FIG. 17B may be performed by aprocessor of a surgical robotic system. For convenience, the method 1750is described as performed by the processor of the system.

The method 1750 may be performed as a part of a setup procedure for amedical procedure, such as the medical procedure 1700 illustrated inFIG. 17A. In certain implementations, the method 1750 may be performedduring block 1710 to provide an indication of a boundary during movementof a robotic arm.

The method 1750 begins at block 1751. At block 1755 the processordetermines a minimum stroke length of the robotic arm that allowsadvancing of a medical instrument by the robotic arm to reach a targetregion. The processor may determine the minimum stroke length based on amapping of an anatomy of a patient. The mapping may comprise dataregarding (i) the target region within the anatomy and (ii) a path froman access point of the patient to the target region. The medicalinstrument may be advanced by the robotic arm to reach the target regionfrom the access point via the path.

At block 1760, the processor determines a boundary for an initial poseof the robotic arm based on the minimum stroke length and the mapping.At block 1765, the processor, during an arm setup phase prior toperforming a procedure, provides an indication of the boundary duringmovement of the robotic arm. The method 1750 ends at block 1770.

Since the structure of the medical instrument and the robotic arms areknown prior to performing the medical procedure, the boundary can bedetermined offline (e.g., prior to the arm setup phase). However,depending on the complexity of the medical procedure and the medicalinstrument, the determination of the boundary may require a significantamount of computational resources. In certain cases, the computation maytake on the order of hours to complete. An example of the considerationsfor defining the initial pose boundary will be described in connectionwith FIG. 18.

FIG. 18 illustrates an embodiment of a bronchoscope which may be used inaccordance with aspects of this disclosure. As shown in FIG. 18, thebronchoscope 200 may include two telescoping parts—namely, a sheath 210and a leader 220. For example, the leader 220 may be the part thatcomprises the camera/vision device, the EM sensors, and the workingchannel through which other instruments can be inserted. The sheath 210may include a base 211, configured to be coupled to an IDM of a firstrobotic arm, and an elongated shaft 213 attached to the base 211.Similarly, the leader 220 may include a base 221, configured to becoupled to an IDM of a second robotic arm, and an elongated shaft 223attached to the base 221. The first and the second robotic arms may beconfigured to advance the sheath 210 and the leader 220, respectively.The elongated shaft 213 of the sheath 210 comprises a working channelthrough which the elongated shaft 223 of the leader 220 is configured tobe inserted. Each of the distal ends of each of the elongated shafts 213and 223 may comprise an articulating section configured to be bent viatension applied to tendons arranged along (or inside) the walls of thecorresponding elongated shafts.

In the example illustrated in FIG. 18, the elongated shaft 213 of thesheath 210 has a length of about 683 mm. As such, the maximum strokelength of the elongated shaft 213 is about 683 mm minus the workinglength of the patient introducer. If the working length of the patientintroducer is about 150 mm, just as an example and not a limitation, themaximum stroke length of the elongated shaft 213 of the sheath 210 maybe about 533 mm. Further, the elongated shaft 223 of the leader 220 maybe about 930 mm in this example. Here, the leader 220 may be extendedabout 130 mm past the distal end of the sheath 210 to allow the distalend of the leader 220 to access the target site, e.g., to perform amedical procedure. Accordingly, the maximum stroke length of the leader220 may be about 663 mm in one example.

However, the respective maximum stroke lengths of the sheath 210 andleader 220 may be reduced based on the setup positioning of the cart androbotic arms. For example, if the first robotic arm (e.g., the armattached to the sheath 210) reaches its maximum extension prior tocontacting the patient introducer, the first robotic arm will not ableto further insert the sheath 210, reducing the achievable stroke length.In another example, after aligning the first robotic arm with thepatient introducer at an initial pose (e.g., in a partially extendedpose), the first robotic arm is retracted to a load instruments pose.However, the distance between the load instruments pose and the initialpose may not be sufficient to achieve the full insertion of the sheath210. Similar considerations affect the achievable stroke length of theleader 220.

Another factor which may limit the achievable stroke length of thesheath 210 and/or the leader 220 include potential collisions betweenone or more of the robotic arm(s) and other objects present in theoperating environment. For example, if the IDM of the second robotic arm(attached to the leader) is prevented from moving into contact with theIDM of the first robotic arm, the leader 220 will lose a portion of thestroke length of the leader 220 past the distal end of the sheath 210.

FIG. 19 is a flow-chart which illustrates another example of a setupprocedure for a bronchoscopy procedure in accordance with aspects ofthis disclosure. The method 1900 may begin after a user has positioned acart adjacent to a patient for alignment of one or more robotic arms ofthe cart with the patient. The method 1900 begins at block 1901. Atblock 1905, the system receives an admittance mode input from a user.One of robotic arms may include an admittance button (e.g., on or nearthe IDM) to allow the user to transition into admittance mode. Asdescribed above, admittance mode may allow the user to apply a forcedirectly to a portion of the robotic arm as input for moving the arm. Incertain implementations, each of the robotic arms may maintain asubstantially constant separation and relative orientation as the usermoves a first robotic arm (of the one or more robotic arms) inadmittance mode.

At block 1910, the system provides an indication of a boundary to theuser. The boundary may be an area in which the user is permitted to movethe first robotic arm in the admittance mode. The boundary may bedetermined by the system such that when the first robotic arm is alignedwith the an access point within the boundary, the stroke lengths of eachof the sheath and the leader are above a predetermined threshold (alsoreferred to as “stroke length threshold” herein). In one example, theaccess point may comprise a patient introducer which may be installed,e.g., in the user's mouth when performing a bronchoscopy procedure.However, in other medical procedures, the access point may compriseanother device designed to guide the medical instrument into the uservia the access point. The access point may also be a natural orifice(e.g., the patient's mouth) without a device installed therein. In otherembodiments, the access point may be small incision(s) which allowinstrument(s) to access the patient's anatomy in a minimally invasivemanner.

Depending on the embodiment, the system may provide the indication ofthe boundary via at least one of: a haptic indication, a visualindication, and an audio indication. For example, the system may providea visual indication of the position of the first robotic arm within aboundary via a display to the user. In another embodiment, a speaker maybe used to indicate the distance to the closest portion of the boundaryor may provide an indication that the user has moved the first roboticarm within a threshold distance of the boundary to provide a warningthat that the first robotic arm is approaching the boundary.

In another embodiment, the system may provide a haptic indication of theboundary to the user. For example, the motors in the first robotic armmay freely allow the user to move the first robotic arm within theboundary, but may prevent movement of the first robotic arm outside ofthe boundary. In certain implementations, this may feel to the user likethe IDM of the first robotic arm is hitting an invisible wall.Alternatively, the first robotic arm may increase a simulated resistanceto movement as the first robotic arm approaches the boundary and preventfurther movement once the IDM reaches the boundary. Accordingly, thesystem may limit the movement of the first robotic arm within theboundary area during the arm setup phase. In other embodiments, such aswhen the boundary is represented by a volume, the system may limit themovement of the first robotic arm within the boundary volume during thearm setup phase.

In certain embodiments, the boundary is a two-dimensional area and onlylimits movement within the plane of the boundary. For example, in abronchoscopy embodiment, the vertical (Z-axis) of the IDM is alignedwith a corresponding feature of the patient introducer. This height isgenerally fixed during the procedure and the insertion of thebronchoscope into the patient does not require any substantial movementin the Z-axis. Thus, changes in the Z-axis of the first robotic armduring the arm setup phase may not significantly affect the achievablestroke length. In these embodiments, the boundary can be thus be definedsolely in the X-Y plane, allowing freedom of movement in the Z-axis.

However, other medical procedures and/or robotic system configurationsmay include movement of one or more robotic arms in the Z-axis duringthe medical procedure. In these embodiments, the boundary may be definein three dimensions including in the Z-axis.

Returning to the method 1900, at block 1915, the system may receive ordetect an input from the user overriding the boundary previouslyprovided to the user. For example, the cart may not have been placed inthe ideal position prior to starting the arm setup phase, and thus, theaccess point (e.g., a patient introducer) may not be within theboundary. When the access point cannot be reached by the IDM of thefirst robotic arm, the user may wish to override the boundary todetermine whether the first robotic arm can be aligned with the accesspoint outside of the boundary. The user may override the boundary byinputting an override command to the system via, for example, anoverride input or another input/output device (e.g., a touchscreendisplay) coupled to the system.

In response to detecting an input to not override the boundary, at block1920, the system aligns the first robotic arm with the access pointbased on input received from the user. For example, the user may apply aforce to the first robotic arm in admittance mode and the system mayalter the position of the first robotic arm using the force as an input.This alignment may involve matching a marking, or other alignmentdevice, on the IDM of the first robotic arm with a corresponding markingor alignment device on the access point (e.g., the patient introducer).Examples of the alignment device include corresponding physical memberson the IDM and patient introducer which can be mated together,electronic communication devices such as an RFID tag/reader, positionaltracking systems (which may be based on optical and/or acoustictechnology), etc. After the first robotic arm is aligned with the accesspoint, at block 1925, the system receives a load instrument pose inputfrom the user. This input may indicate that the user has completed thealignment step and is ready to load the instrument(s) (e.g., the sheathand leader of the bronchoscope) onto the robotic arm(s).

At block 1945, the system moves the first robotic arm to the loadinstruments pose so that a user may load a medical instrument onto thefirst robotic arm.

In response to detecting an input from the user to override theboundary, at block 1930, the system aligns the robotic arm with theaccess point based on input received from the user. For example, theuser may apply a force to the first robotic arm in admittance mode andthe system may alter the position of the first robotic arm using theforce as an input. In this case, the user may move the first robotic armoutside of the boundary previously provided to the user in block 1910.At block 1935, the system calculates the achievable stroke length andprovides an indication of the calculated result to the user. In oneimplementation, the system may calculate and provide the indication tothe user in “real-time.” For example, the system may detect the movementof the first robotic arm at a sampling frequency. The system may thendetermine a position of the first robotic arm based on the detectedmovement of the first robotic arm. The system may then simulate, basedon the position of the first robotic arm, an achievable stroke length ofthe first robotic arm that facilitates advancing the medical instrumentinto the patient. One technique for simulating the achievable strokelength will be described in greater detail below in connection with FIG.20.

There are a number of different techniques which may be used to providethe indication of the achievable stroke length to the user. In oneimplementation, the system may provide a numerical value to the userthat is representative of the simulated achievable stroke length. Inanother implementation, the system may store (e.g., in memory) therequired stroke length for performing a specific medical procedure onthe patient. In certain embodiments, the required stroke length may bethe minimum stroke length of the robotic arm that allows advancing of amedical instrument by the robotic arm to reach a target region from thepatient's access point via a predetermined path therebetween.

For example, prior to the arm setup phase, an image of the patient'sluminal network may have been captured. For certain medical procedures,a pre-operative mapping of the patient's luminal network may have beenperformed through the use of the collection of low dose CT scans. Basedon the mapping, the system may determine a path from the selected accesspoint to the target destination and calculate the stroke length requiredto reach the target destination via the determined path. The system maythen compare the required stroke length to the achievable stroke lengthand provide an indication of whether the target destination can bereached to the user based on the pose of the first robotic arm. Forexample, the system may determine whether the achievable stroke lengthis greater than or equal to the minimum stroke length and provide anindication of whether the achievable stroke length is greater than orequal to the minimum stroke length. Depending on the computationalbandwidth of the system, the system may be able to continually updatethe simulation of the procedure and provide the indication of whetherthe desired target destination can be reached as the user moves thefirst robotic arm in admittance mode.

In certain embodiment, the system may determine whether the achievablestroke length is greater than or equal to the minimum stroke length inresponse to the user releasing the admittance mode button. For example,while in admittance mode, the user may still be moving the robotic arminto alignment, and thus, the initial pose of the robotic arm may not beset while the admittance button is depressed. After the user hasreleased the admittance button, the system may infer that the pose ofthe first robotic arm is in alignment with the access point, and thus,may compare the achievable stroke length to the minimum stroke lengththereafter. Thus, in certain implementations, the system may onlyprovide or update the indication of whether the target destination canbe reached in response to the user releasing the admittance mode button.

At block 1940, the system receives an input from the user accepting theprovided stroke length. For example, the system may be configured toreceive using input that is indicative of whether the providedachievable stroke length is acceptable to the user. When the useraccepts the provided achievable stroke length, the system may receive aninput (e.g., a user intput) including an instruction for the system tomove into the load instruments pose at block 1925. If the input receivedby the system indicates that the user has not accepted the achievablestroke length, the method ends at block 1950 and the system may displayinstructions to the user to reposition the cart to increase theachievable stroke length. However, in other embodiments, the system maybe able to determine the a direction in which the first robotic arm maybe moved to adjust the achievable stroke length such that the medicalprocedure can be performed; in such embodiments, the system maydetermine that the achievable stroke length is less than the minimumstroke length, calculate a direction from the current position to theboundary, and provide an indication of the direction from the currentposition to the boundary.

C. Simulation of Achievable Stroke Length

In certain embodiments, to determine the achievable stroke length for aninitial position of one or more robotic arms, the system may perform asimulation of the medical procedure. For example, a given medicalprocedure may include a set sequence of movements of the associatedrobotic arms that depend on the specific procedure involved. Thesequence of movements may also depend on the characteristics of thepatient depending on the medical procedure.

FIG. 20 is a flow-chart which illustrates an example methodology forsimulating a medical procedure in accordance with aspects of thisdisclosure. In particular, the FIG. 20 embodiment relates to abronchoscopy procedure. The specifics of a simulation will depend on themedical procedure being performed and the corresponding configuration ofthe surgical system including the number of robotic arms involved in theprocedure, the alignment process, any additional device attached orinstalled in the patient, etc.

The method 2000 starts at block 2001. At block 2005, the systemsimulates full insertion of the leader into the sheath. Since theposition of the sheath robotic arm is assumed to be in alignment withthe access point (e.g., a patient introducer) for the purposes of thesimulation, block 2005 corresponds to full insertion of both the sheathand leader into the patient. At block 2010, the system determines theachievable leader stroke length based on the results of the simulatedinsertion in block 2005. That is, if the simulation results in some sortof collision or other barrier to the leader being fully inserted intothe sheath, the simulated achievable stroke length of the leader may beless than the maximum achievable leader stroke length under idealconditions.

At block 2015, the system simulates retracting the leader from thesheath working channel. This may involve simulating the retraction ofthe leader to the initial pose of the leader robotic arm prior to theblock 2005 simulation. At block 2020, the system simulates the fullretraction of both the sheath and the leader from the patient. Here,full retraction may refer to retracting the sheath and leader fully fromthe patient, but leaving the sheath and leader in the access point. Atblock 2025, the system determines the achievable stroke length of thesheath based on the simulated results of block 2020. For example, if thesystem determines that a collision or other barrier would prevent thesheath being fully retracted from the patient, the simulated achievablestroke length of the sheath may be less than the maximum achievablesheath stroke length under ideal conditions.

At block 2030, the system compares the simulated achievable leader andsheath strokes lengths with the required stroke length(s) for theprocedure. As described above, the system may be able to calculate therequired stroke lengths for the procedure based on a pre-operativemapping of the patient's luminal network. The method 200 ends at block2035.

The method 2000 described in connection with FIG. 20 includes asimulation of a bronchoscopy procedure. However, the sequence ofmovements during a specific medical procedure may be altered from thesequence described in connection with FIG. 20 depending on therequirements for performing the procedure. Similarly, the sequence ofevents may be simulated in orders other than shown in FIG. 20 as long asthe achievable stroke lengths of the leader and sheath can be simulated.

In another example, the system may simulate the movement of first andsecond robotic arms in the same sequence as performed during aprocedure. For example, the system may determine at least one firstmovement of the first robotic arm that facilitates advancing the sheathfrom the access point to the target region via the path. The system mayalso determine at least one second movement of the second robotic armthat facilitates advancing the leader through the sheath to the targetregion. Thereafter, the system may simulate the at least one firstmovement and the at least one second movement at a position of the firstand second robotic arms and calculate at least one achievable strokelength of the first and second robotic arms based on the simulation.

Although the method 2000 has been described in connection with abronchoscopy example, aspects of this disclosure can also simulate theachievable stroke length(s) for one or more robotic arms for other typesof medical or surgical procedures. In certain implementations, therobotic system may be configurable to perform one of a plurality ofsurgical procedures. In these implementations, the system may beconfigured to receive an input indicative of a surgical procedure forthe patient. Based on the received surgical procedure input, the systemmay determine at least one target movement of the first robotic arm thatfacilitates the advancing of the medical instrument from the accesspoint to the target region via the path and performing the surgicalprocedure at the target region. This may be based on the surgical ormedical instrument used to perform the procedure as well as the numberof robotic arms required to manipulate the instrument. The system mayalso simulate the at least one target movement at the position of thefirst robotic arm and calculate an achievable stroke length of the firstrobotic arm based on the simulated movement.

The system may determine that the simulated movement would result in thefirst robotic arm colliding with an object. The object may be anotherportion of the surgical robotic system, such as another robotic arm, thecart, etc. or may be another object within the operating environment. Inorder to simulate collisions with other objects, the position and shapeof object(s) within the operating environment may be programmed into thesystem's memory. For example, a C-arm may be used in various medicalprocedures and may be located within the working area of the roboticarms. Thus, the system may simulate whether a given procedure, based onthe selected initial pose, would result in collision with the C-arm. Thecalculating of the achievable stroke length may be further based on thedetermination that the simulated movement would result in the firstrobotic arm colliding with the object and the determination of whetherthe simulated movement would result in the first robotic arm being fullyextended.

The full extension of one or more of the robotic arms may also limit thestroke length of the corresponding arms. Thus, during simulation, thesystem may also determine whether the simulated movement would result inone of the robotic arms being fully extended. The system may furthercalculate of the achievable stroke length based on the determinationthat the simulated movement would result in the one of the robotic armsbeing fully extended.

In alternative implementations, rather than calculating or simulatingthe achievable stroke length in real-time, the system may store a strokelength validation module in memory that defines an achievable strokelength of the first robotic arm for each of a plurality of initial posesof the first robotic arm. The stroke length validation module maycomprise a database or procedure, stored in memory, which relates valuesof initial poses of the first robotic arm to corresponding achievablestroke lengths. In certain embodiments, the stroke length validationmodule may comprise a look-up table, or other data structure, thatstores an achievable stroke length for each of a plurality of initialposes of the first robotic arm. In other embodiments, the stroke lengthvalidation module may comprise a technique for calculating theachievable stroke length for a given initial pose of the robotic arm.

This may be practical for medical procedures that are well defined andin which there are no additional factors (e.g., object collisions) thatcan limit stroke length besides the initial pose of the first roboticarm. These implementations may include during the arm setup phasedetecting a movement of the first robotic arm to the initial pose andretrieve an achievable stroke length from the stroke length validationmodule based on the initial pose. The system may then determine whetherthe achievable stroke length is greater than or equal to the minimumstroke length and provide an indication of whether the achievable strokelength is greater than or equal to the minimum stroke length. The systemmay also retrieve the achievable stroke length from the stroke lengthvalidation module in response to detecting a user-initiated event, suchas a user input instructing the system to use the stroke lengthvalidation module in determining the achievable stroke length.

3. Implementing Systems and Terminology

Implementations disclosed herein provide systems, methods and apparatusfor determining whether an initial pose of a robotic arm will provide asufficient stroke length for a medical procedure. This determination mayinclude, in certain embodiments, a simulation of the procedure todetermine the achievable stroke length of the robotic arm.

It should be noted that the terms “couple,” “coupling,” “coupled” orother variations of the word couple as used herein may indicate eitheran indirect connection or a direct connection. For example, if a firstcomponent is “coupled” to a second component, the first component may beeither indirectly connected to the second component via anothercomponent or directly connected to the second component.

The calculation and determination functions for simulating anddetermining stroke length described herein may be stored as one or moreinstructions on a processor-readable or computer-readable medium. Theterm “computer-readable medium” refers to any available medium that canbe accessed by a computer or processor. By way of example, and notlimitation, such a medium may comprise random access memory (RAM),read-only memory (ROM), electrically erasable programmable read-onlymemory (EEPROM), flash memory, compact disc read-only memory (CD-ROM) orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that can be used to store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. It should be noted that a computer-readablemedium may be tangible and non-transitory. As used herein, the term“code” may refer to software, instructions, code or data that is/areexecutable by a computing device or processor.

The methods disclosed herein comprise one or more steps or actions forachieving the described method. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isrequired for proper operation of the method that is being described, theorder and/or use of specific steps and/or actions may be modifiedwithout departing from the scope of the claims.

As used herein, the term “plurality” denotes two or more. For example, aplurality of components indicates two or more components. The term“determining” encompasses a wide variety of actions and, therefore,“determining” can include calculating, computing, processing, deriving,investigating, looking up (e.g., looking up in a table, a database oranother data structure), ascertaining and the like. Also, “determining”can include receiving (e.g., receiving information), accessing (e.g.,accessing data in a memory) and the like. Also, “determining” caninclude resolving, selecting, choosing, establishing and the like.

The phrase “based on” does not mean “based only on,” unless expresslyspecified otherwise. In other words, the phrase “based on” describesboth “based only on” and “based at least on.”

The previous description of the disclosed implementations is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these implementations will bereadily apparent to those skilled in the art, and the generic principlesdefined herein may be applied to other implementations without departingfrom the scope of the invention. For example, it will be appreciatedthat one of ordinary skill in the art will be able to employ a numbercorresponding alternative and equivalent structural details, such asequivalent ways of fastening, mounting, coupling, or engaging toolcomponents, equivalent mechanisms for producing particular actuationmotions, and equivalent mechanisms for delivering electrical energy.Thus, the present invention is not intended to be limited to theimplementations shown herein but is to be accorded the widest scopeconsistent with the principles and novel features disclosed herein.

What is claimed is:
 1. A robotic system for performing a medicalprocedure, comprising: a first robotic arm configured to manipulate amedical instrument; a processor; and a memory storing a mapping of ananatomy of a patient, the mapping comprising data regarding (i) a targetregion within the anatomy and (ii) a path from an access point of thepatient to the target region, the memory further storingcomputer-executable instructions, that when executed, cause theprocessor to: determine a minimum stroke length of the first robotic armthat allows advancing of the medical instrument by the first robotic armto reach the target region from the access point via the path, determinea boundary for an initial pose of the first robotic arm based on theminimum stroke length and the mapping, and during an arm setup phaseprior to performing the procedure, provide an indication of the boundaryduring movement of the first robotic arm.
 2. The system of claim 1,wherein the memory further comprises computer-executable instructions,that when executed, cause the processor to: provide the indication ofthe boundary via at least one of: a haptic indication, a visualindication, and an audio indication.
 3. The system of claim 1, wherein:the boundary comprises a boundary area; and the memory further comprisescomputer-executable instructions to cause the processor to limit themovement of the first robotic arm within the boundary area during thearm setup phase.
 4. The system of claim 1, wherein the memory furthercomprises computer-executable instructions, that when executed, causethe processor to: limit movement of the first robotic arm to within theboundary during the arm setup phase.
 5. The system of claim 4, whereinthe memory further comprises computer-executable instructions, that whenexecuted, cause the processor to: detect an input to override the limit,detect a movement of the first robotic arm, determine a position of thefirst robotic arm based on the detected movement of the first roboticarm, and simulate, based on the position of the first robotic arm, anachievable stroke length of the first robotic arm that facilitatesadvancing the medical instrument into the patient.
 6. The system ofclaim 1, wherein the memory further comprises computer-executableinstructions, that when executed, cause the processor to: receive aninput indicative of a surgical procedure for the patient; determine,based on the surgical procedure, at least one target movement of thefirst robotic arm that facilitates the advancing of the medicalinstrument from the access point to the target region via the path andperforming the surgical procedure at the target region, simulate the atleast one target movement at the position of the first robotic arm, andcalculate an achievable stroke length of the first robotic arm based onthe simulated movement.
 7. The system of claim 6, wherein the memoryfurther comprises computer-executable instructions, that when executed,cause the processor to: determine that the simulated movement wouldresult in the first robotic arm colliding with an object, wherein thecalculating of the achievable stroke length is further based on thedetermination that the simulated movement would result in the firstrobotic arm colliding with the object and the determination of whetherthe simulated movement would result in the first robotic arm being fullyextended.
 8. The system of claim 1, wherein the memory further comprisescomputer-executable instructions, that when executed, cause theprocessor to: determine that the simulated movement would result in thefirst robotic arm being fully extended, wherein the calculating of theachievable stroke length is further based on the determination that thesimulated movement would result in the first robotic arm being fullyextended.
 9. The system of claim 5, wherein the memory further comprisescomputer-executable instructions, that when executed, cause theprocessor to: determine that the achievable stroke length is less thanthe minimum stroke length, calculate a direction from the currentposition to the boundary, and provide an indication of the directionfrom the current position to the boundary.
 10. The system of claim 1,wherein: the medical instrument comprises a sheath; the system furthercomprises a second robotic arm configured to advance a leader throughthe sheath; and the memory further comprises computer-executableinstructions, that when executed, cause the processor to: determine atleast one first movement of the first robotic arm that facilitatesadvancing the sheath from the access point to the target region via thepath, determine at least one second movement of the second robotic armthat facilitates advancing the leader through the sheath to the targetregion, simulate the at least one first movement and the at least onesecond movement at a position of the first and second robotic arms, andcalculate at least one achievable stroke length of the first and secondrobotic arms based on the simulation.
 11. A non-transitory computerreadable storage medium having stored thereon instructions that, whenexecuted, cause at least one computing device to: determine a minimumstroke length of a first robotic arm that allows advancing of a medicalinstrument by the first robotic arm to a target region based on amapping of an anatomy of a patient, the mapping comprising dataregarding (i) the target region within the anatomy and (ii) a path froman access point of the patient to the target region, the medicalinstrument advanced to reach the target region from the access point viathe path; determine a boundary for an initial pose of the first roboticarm based on the minimum stroke length and the mapping; and during anarm setup phase prior to performing a procedure, provide an indicationof the boundary during movement of the first robotic arm.
 12. Thenon-transitory computer readable storage medium of claim 11, furtherhaving stored thereon instructions that, when executed, cause at leastone computing device to: provide the indication of the boundary via atleast one of: a haptic indication, a visual indication, and an audioindication.
 13. The non-transitory computer readable storage medium ofclaim 11, the boundary comprises a boundary area, the non-transitorycomputer readable storage medium further having stored thereoninstructions that, when executed, cause at least one computing deviceto: limit the movement of the first robotic arm within the boundary areaduring the arm setup phase.
 14. The non-transitory computer readablestorage medium of claim 11, further having stored thereon instructionsthat, when executed, cause at least one computing device to: limitmovement of the first robotic arm to within the boundary during the armsetup phase.
 15. The non-transitory computer readable storage medium ofclaim 14, further having stored thereon instructions that, whenexecuted, cause at least one computing device to: detect an input tooverride the limit, detect a movement of the first robotic arm,determine a position of the first robotic arm based on the detectedmovement of the first robotic arm, and simulate, based on the positionof the first robotic arm, an achievable stroke length of the firstrobotic arm that facilitates advancing the medical instrument into thepatient.
 16. The non-transitory computer readable storage medium ofclaim 11, further having stored thereon instructions that, whenexecuted, cause at least one computing device to: receive an inputindicative of a surgical procedure for the patient; determine, based onthe surgical procedure, at least one target movement of the firstrobotic arm that facilitates the advancing of the medical instrumentfrom the access point to the target region via the path and performingthe surgical procedure at the target region, simulate the at least onetarget movement at the position of the first robotic arm, and calculatean achievable stroke length of the first robotic arm based on thesimulated movement.
 17. The non-transitory computer readable storagemedium of claim 16, further having stored thereon instructions that,when executed, cause at least one computing device to: determine thatthe simulated movement would result in the first robotic arm collidingwith an object, wherein the calculating of the achievable stroke lengthis further based on the determination that the simulated movement wouldresult in the first robotic arm colliding with the object and thedetermination of whether the simulated movement would result in thefirst robotic arm being fully extended.
 18. The non-transitory computerreadable storage medium of claim 17, further having stored thereoninstructions that, when executed, cause at least one computing deviceto: determine that the simulated movement would result in the firstrobotic arm being fully extended, wherein the calculating of theachievable stroke length is further based on the determination that thesimulated movement would result in the first robotic arm being fullyextended.
 19. The non-transitory computer readable storage medium ofclaim 15, further having stored thereon instructions that, whenexecuted, cause at least one computing device to: determine that theachievable stroke length is less than the minimum stroke length,calculate a direction from the current position to the boundary, andprovide an indication of the direction from the current position to theboundary.
 20. The non-transitory computer readable storage medium ofclaim 11, wherein: the medical instrument comprises a sheath; the systemfurther comprises a second robotic arm configured to advance a leaderthrough the sheath; and the non-transitory computer readable storagemedium further has stored thereon instructions that, when executed,cause at least one computing device to: determine at least one firstmovement of the first robotic arm that facilitates advancing the sheathfrom the access point to the target region via the path, determine atleast one second movement of the second robotic arm that facilitatesadvancing the leader through the sheath to the target region, simulatethe at least one first movement and the at least one second movement ata position of the first and second robotic arms, and calculate at leastone achievable stroke length of the first and second robotic arms basedon the simulation.
 21. A method of positioning a first robotic arm,comprising: determining a minimum stroke length of the first robotic armthat allows advancing of a medical instrument by the first robotic armto reach a target region based on a mapping of an anatomy of a patient,the mapping comprising data regarding (i) the target region within theanatomy and (ii) a path from an access point of the patient to thetarget region, the medical instrument advanced to reach the targetregion from the access point via the path; determine a boundary for aninitial pose of the first robotic arm based on the minimum stroke lengthand the mapping; and during an arm setup phase prior to performing aprocedure, provide an indication of the boundary during movement of thefirst robotic arm.
 22. The method of claim 21, further comprising:providing the indication of the boundary via at least one of: a hapticindication, a visual indication, and an audio indication.
 23. The methodof claim 21, wherein the boundary comprises a boundary area, the methodfurther comprising: limiting the movement of the first robotic armwithin the boundary area during the arm setup phase.
 24. The method ofclaim 21, further comprising: limiting movement of the first robotic armto within the boundary during the arm setup phase.
 25. The method ofclaim 24, further comprising: detecting an input to override the limit,detecting a movement of the first robotic arm, determining a position ofthe first robotic arm based on the detected movement of the firstrobotic arm, and simulating, based on the position of the first roboticarm, an achievable stroke length of the first robotic arm thatfacilitates advancing the medical instrument into the patient.
 26. Themethod of claim 21, further comprising: receiving an input indicative ofa surgical procedure for the patient; determining, based on the surgicalprocedure, at least one target movement of the first robotic arm thatfacilitates the advancing of the medical instrument from the accesspoint to the target region via the path and performing the surgicalprocedure at the target region, simulating the at least one targetmovement at the position of the first robotic arm, and calculating anachievable stroke length of the first robotic arm based on the simulatedmovement.
 27. The method of claim 26, further comprising: determiningthat the simulated movement would result in the first robotic armcolliding with an object, wherein the calculating of the achievablestroke length is further based on the determination that the simulatedmovement would result in the first robotic arm colliding with the objectand the determination of whether the simulated movement would result inthe first robotic arm being fully extended.
 28. The method of claim 27,further comprising: determining that the simulated movement would resultin the first robotic arm being fully extended, wherein the calculatingof the achievable stroke length is further based on the determinationthat the simulated movement would result in the first robotic arm beingfully extended.
 29. The method of claim 25, further comprising:determining that the achievable stroke length is less than the minimumstroke length, calculating a direction from the current position to theboundary, and providing an indication of the direction from the currentposition to the boundary.
 30. The method of claim 21, wherein: themedical instrument comprises a sheath; the system further comprises asecond robotic arm configured to advance a leader through the sheath;and the method further comprises: determining at least one firstmovement of the first robotic arm that facilitates advancing the sheathfrom the access point to the target region via the path, determining atleast one second movement of the second robotic arm that facilitatesadvancing the leader through the sheath to the target region, simulatingthe at least one first movement and the at least one second movement ata position of the first and second robotic arms, and calculating atleast one achievable stroke length of the first and second robotic armsbased on the simulation.